Last data update: May 13, 2024. (Total: 46773 publications since 2009)
Records 1-3 (of 3 Records) |
Query Trace: Hadgu A[original query] |
---|
Bias due to composite reference standards in diagnostic accuracy studies
Schiller I , van Smeden M , Hadgu A , Libman M , Reitsma JB , Dendukuri N . Stat Med 2015 35 (9) 1454-70 Composite reference standards (CRSs) have been advocated in diagnostic accuracy studies in the absence of a perfect reference standard. The rationale is that combining results of multiple imperfect tests leads to a more accurate reference than any one test in isolation. Focusing on a CRS that classifies subjects as disease positive if at least one component test is positive, we derive algebraic expressions for sensitivity and specificity of this CRS, sensitivity and specificity of a new (index) test compared with this CRS, as well as the CRS-based prevalence. We use as a motivating example the problem of evaluating a new test for Chlamydia trachomatis, an asymptomatic disease for which no gold-standard test exists. As the number of component tests increases, sensitivity of this CRS increases at the expense specificity, unless all tests have perfect specificity. Therefore, such a CRS can lead to significantly biased accuracy estimates of the index test. The bias depends on disease prevalence and accuracy of the CRS. Further, conditional dependence between the CRS and index test can lead to over-estimation of index test accuracy estimates. This commonly-used CRS combines results from multiple imperfect tests in a way that ignores information and therefore is not guaranteed to improve over a single imperfect reference unless each component test has perfect specificity, and the CRS is conditionally independent of the index test. When these conditions are not met, as in the case of C. trachomatis testing, more realistic statistical models should be researched instead of relying on such CRSs. |
One in 16 quadrillion: importance of blinding in research reproducibility
Hadgu A , Nordbo SA , Skjeldestad FE . Epidemiology 2012 23 (3) 503-4 A hallmark of good science is reproducibility. If a study is not reproducible, something may have gone wrong. Recent science policy essays have addressed research reproducibility and statistical methods to improve it.1 The importance of reproducibility is illustrated in a Lancet article that claimed a breakthrough in detection of ovarian cancer through the use of mass spectrometry.2 The authors stated that this test could establish whether a woman had ovarian cancer with almost perfect accuracy (100% sensitivity and 95% specificity).2 However, reanalysis of these data raised serious doubts about the reproducibility of this work.3 Recently, Duke University had to cancel 3 clinical trials because they were based on published research that could not be reproduced.1 Here, we demonstrate how elementary statistics can help assess the reproducibility concerns of published manuscripts. We show this in the context of Chlamydia trachomatis test-retest settings. | Datta et al4 tested Chlamydia trachomatis and Neisseria gonorrhoeae specimens using the ligase chain reaction (LCx) assay. Of the 6632 samples tested, 241 were positive for chlamydia and 36 were positive for gonorrhea. The authors stated that “… specimens positive for C. trachomatis or N. gonorrhoeae were retested from the original urine specimen by using the same assay for detection for the purposes of this survey. No retests yielded discrepant results. Specimens with negative results were not retested.”4 In other words, the percentage of positive results confirmed by this laboratory was 100%. Here, we demonstrate that a 100% confirmation of chlamydia-positive results is statistically implausible and is inconsistent with previously published studies. |
Evaluation of screening tests for detecting Chlamydia trachomatis: bias associated with the patient-infected-status algorithm
Hadgu A , Dendukuri N , Wang L . Epidemiology 2012 23 (1) 72-82 In recent years, the evaluation of nucleic acid amplification tests (NAATs) for detecting Chlamydia trachomatis and Neisseria gonorrhea is based on a methodology called the patient-infected-status algorithm (PISA). In the simplest version of PISA, 4 test-specimen combinations (comparator tests) are used to define the gold standard. If a person shows a positive result by any 2 or more of these 4 comparator tests, the person is classified as infected; otherwise, the person is considered to be uninfected. A new test is then compared with this diagnostic algorithm. PISA-based sensitivity and specificity estimates of nucleic acid amplification tests have been published in the medical and microbiologic literature and have been included in FDA-approved package inserts of NAATs for detecting C. trachomatis. Using simulations, we compare 2 versions of the patient-infected-status algorithm with latent-class models and an imperfect gold standard. We show that the PISA can produce highly biased test-performance parameter estimates. In a series of simulated scenarios, none of the 95% confidence intervals for PISA-based estimates of sensitivity and prevalence contained the true values. In addition, the PISA-based estimates of sensitivity and specificity change markedly as the true prevalence changes. We recommend that PISA should not be used for estimating the sensitivity and specificity of tests. |
- Page last reviewed:Feb 1, 2024
- Page last updated:May 13, 2024
- Content source:
- Powered by CDC PHGKB Infrastructure